Factors influencing visual attention switch in multi-display user interfaces: a survey

Multi-display User Interfaces (MDUIs) enable people to take advantage of the different characteristics of different display categories. For example, combining mobile and large displays within the same system enables users to interact with user interface elements locally while simultaneously having a large display space to show data. Although there is a large potential gain in performance and comfort, there is at least one main drawback that can override the benefits of MDUIs: the visual and physical separation between displays requires that users perform visual attention switches between displays. In this paper, we present a survey and analysis of existing data and classifications to identify factors that can affect visual attention switch in MDUIs. Our analysis and taxonomy bring attention to the often ignored implications of visual attention switch and collect existing evidence to facilitate research and implementation of effective MDUIs.Postprin

Opportunistic visualization with iVoLVER

Proposed as 'data analysis anywhere, anytime, from anything', Opportunistic Information Visualization (Opportu-Vis) [1] seeks to provide analytical support in scenarios where the data of interest is not explicitly available and has to be retrieved from digital artifacts that are not traditionally used as data sources. Examples include raster images, web pages, vector files, and photographs. This showpiece presents how iVoLVER, the Interactive Visual Language for Visualization Extraction and Reconstruction, provides support in such settings. We briefly describe the overall construction approach of the tool in scenarios where different digital artifacts are used to compose interactive visuals. All of this becomes possible by using the data extraction capabilities of iVoLVER together with the elements of its visual language.Postprin

Reading small scalar data fields: color scales vs. Detail on Demand vs. FatFonts

We empirically investigate the advantages and disadvantages of color- and digit-based methods to represent small scalar fields. We compare two types of color scales (one brightness-based and one that varies in hue, saturation and brightness) with an interactive tooltip that shows the scalar value on demand, and with a symbolic glyph-based approach (FatFonts). Three experiments tested three tasks: reading values, comparing values, and finding extrema. The results provide the first empirical comparisons of color scales with symbol-based techniques. The interactive tooltip enabled higher accuracy and shorter times than the color scales for reading values but showed slow completion times and low accuracy for value comparison and extrema finding tasks. The FatFonts technique showed better speed and accuracy for reading and value comparison, and high accuracy for the extrema finding task at the cost of being the slowest for this task.Postprin

Representational transformations : using maps to write essays

This research was supported by NSERC (The Natural Sciences and Engineering Research Council of Canada) RGPIN-2020-04401 and EPSRC (Engineering and Physical Sciences Research Council) EP/T518062/1.Essay-writing is a complex, cognitively demanding activity. Essay-writers must synthesise source texts and original ideas into a textual essay. Previous work found that writers produce better essays when they create effective intermediate representations. Diagrams, such as concept maps and argument maps, are particularly effective. However, there is insufficient knowledge about how people use these intermediate representations in their essay-writing workflow. Understanding these processes is critical to inform the design of tools to support workflows incorporating intermediate representations. We present the findings of a study, in which 20 students planned and wrote essays. Participants used a tool that we developed, Write Reason, which combines a free-form mapping interface with an essay-writing interface. This let us observe the types of intermediate representations participants built, and crucially, the process of how they used and moved between them. The key insight is that much of the important cognitive processing did not happen within a single representation, but instead in the processes that moved between multiple representations. We label these processes `representational transformations'. Our analysis characterises key properties of these transformations: cardinality, explicitness, and change in representation type. We also discuss research questions surfaced by the focus on transformations, and implications for tool designers.Publisher PDFPeer reviewe

Integrating 2D Mouse Emulation with 3D Manipulation for Visualizations on a Multi-Touch Table

We present the Rizzo, a multi-touch virtual mouse that has been designed to provide the fine grained interaction for information visualization on a multi-touch table. Our solution enables touch interaction for existing mouse-based visualizations. Previously, this transition to a multi-touch environment was difficult because the mouse emulation of touch surfaces is often insufficient to provide full information visualization functionality. We present a unified design, combining many Rizzos that have been designed not only to provide mouse capabilities but also to act as zoomable lenses that make precise information access feasible. The Rizzos and the information visualizations all exist within a touch-enabled 3D window management system. Our approach permits touch interaction with both the 3D windowing environment as well as with the contents of the individual windows contained therein. We describe an implementation of our technique that augments the VisLink 3D visualization environment to demonstrate how to enable multi-touch capabilities on all visualizations written with the popular prefuse visualization toolkit.

Bottom-up vs. top-down : trade-offs in efficiency, understanding, freedom and creativity with InfoVis tools

The emergence of tools that support fast-and-easy visualization creation by non-experts has made the benefits of InfoVis widely accessible. Key features of these tools include attribute-level operations, automated mappings, and visualization templates. However, these features shield people from lower-level visualization design steps, such as the specific mapping of data points to visuals. In contrast, recent research promotes constructive visualization where individual data units and visuals are directly manipulated. We present a qualitative study comparing people's visualization processes using two visualization tools: one promoting a top-down approach to visualization construction (Tableau Desktop) and one implementing a bottom-up constructive visualization approach (iVoLVER). Our results show how the two approaches influence: 1) the visualization process, 2) decisions on the visualization design, 3) the feeling of control and authorship, and 4) the willingness to explore alternative designs. We discuss the complex trade-offs between the two approaches and outline considerations for designing better visualization tools.Postprin

VisuaLizations As Intermediate Representations (VLAIR) : an approach for applying deep learning-based computer vision to non-image-based data

We thank the China Scholarship Council (CSC) for financially supporting my PhD study at University of St Andrews, UK, and NSERC Discovery Grant 2020-04401 (Miguel Nacenta).Deep learning algorithms increasingly support automated systems in areas such as human activity recognition and purchase recommendation. We identify a current trend in which data is transformed first into abstract visualizations and then processed by a computer vision deep learning pipeline. We call this VisuaLization As Intermediate Representation (VLAIR) and believe that it can be instrumental to support accurate recognition in a number of fields while also enhancing humans’ ability to interpret deep learning models for debugging purposes or in personal use. In this paper we describe the potential advantages of this approach and explore various visualization mappings and deep learning architectures. We evaluate several VLAIR alternatives for a specific problem (human activity recognition in an apartment) and show that VLAIR attains classification accuracy above classical machine learning algorithms and several other non-image-based deep learning algorithms with several data representations.Publisher PDFPeer reviewe

All across the circle : using auto-ordering to improve object transfer between mobile devices

People frequently form small groups in many social and professional situations: from conference attendees meeting at a coffee break, to siblings gathering at a family barbecue. These ad-hoc gatherings typically form into predictable geometries based on circles or circular arcs (called F-Formations). Because our lives are increasingly stored and represented by data on handheld devices, the desire to be able to share digital objects while in these groupings has increased. Using the relative position in these groups to facilitate file sharing can enable intuitive techniques such as passing or flicking. However, there is no reliable, lightweight, ad-hoc technology for detecting and representing relative locations around a circle. In this paper, we present two systems that can auto-order locations about a circle based on sensors that are standard on commodity smartphones. We tested these systems using an object-passing task in a laboratory environment against unordered and proximity-based systems, and show that our techniques are faster, are more accurate, and are preferred by users.Postprin

Visualization as Intermediate Representations (VLAIR) for human activity recognition

Ambient, binary, event-driven sensor data is useful for many human activity recognition applications such as smart homes and ambient-assisted living. These sensors are privacy-preserving, unobtrusive, inexpensive and easy to deploy in scenarios that require detection of simple activities such as going to sleep, and leaving the house. However, classification performance is still a challenge, especially when multiple people share the same space or when different activities take place in the same areas. To improve classification performance we develop what we call a Visualization as Intermediate Representations (VLAIR) approach. The main idea is to re-represent the data as visualizations (generated pixel images) in a similar way as how visualizations are created for humans to analyze and communicate data. Then we can feed these images to a convolutional neural network whose strength resides in extracting effective visual features. We have tested five variants (mappings) of the VLAIR approach and compared them to a collection of classifiers commonly used in classic human activity recognition. The best of the VLAIR approaches outperforms the best baseline, with strong advantage in recognising less frequent activities and distinguishing users and activities in common areas. We conclude the paper with a discussion on why and how VLAIR can be useful in human activity recognition scenarios and beyond.Postprin